Job Monitor Collection

When the LS Data Director has finished a processing job, it moves the job header XML file, with all the information about the job, to the Archive folder, located under the Data Folder path.

This data can be converted into Monitor Data Jobs named DDArchiveJob and DDStatusWorkJob that can be sent to central database for reporting purposes or to be used with the Web Monitor program that runs off these database tables.

Monitoring tables are: (in Central 17.5 and later these tables have an LSC prefix)

  • DDMonitorHdr
  • DDMonitorLine
  • DDMonitorLocStatus
  • DDScheduler
  • DDSchLog

The Monitor Data Job inserts the data to DDMonitorHdr and DDMonitorLine tables.

  • DDMonitorHdr includes the basic information about the job such as Pack/Job Id, Job status, and create time.
  • DDMonitorLine contains information from the source and destination hosts, such as host status, server messages, connection string, and last registration time. The index ID for the host is -1 for the source host (where the data was pulled from) and 0 and up for destination hosts (where data will be inserted).

The Monitoring Collection setup is needed where the Source data will be pulled from and where Scheduler sends Jobs to, as the statues from the destination DD Hosts are always reported back to the Source Host. If the Scheduler at Head Office is pulling data from the stores with pull jobs, then the Monitor Collection is required at both the Head Office and store DDs. If the Head Office is only sending jobs to stores, then it is only required at the Head Office.

  1. In the DD Configuration Tool, click the Monitor Config tab, and select the Activate check box:

  2. Type in the DD Host of the DD Service you want to send the monitor data to, and the database connection string of the central database in the DB String field where the monitor tables are located. You can copy the connection string from the Distribution Location card in LS Central.

  3. If the Monitor data job needs to be routed via another DD Host, type the routing path in the DD Host.

    Example: If POS is running DD and jobs need to be routed via Store Server, then type STORE;HQSRV into the DD Host field and the Job will be routed from POS via the Store server to Head Quarters.

  4. Click the Plugins button to select what data should be included:

    DD has three default plugins:

    • Archive Data Collector takes the data from the Archive folder. These are all processed jobs with the status Done.
    • Work Data Collector gets the job data from the DD Router's memory. These are all jobs still in processing, waiting for processing or jobs with error.
    • Result Data Collector takes result data from Manual Replication and updates the Monitor data with the results.
  5. Select the plugin you want to run, and type in the Interval how often you want to send data for the plugin. Other plugins can be created that would collect some other information like OS status or Database Server data.

    Note: There are xxx Pre DataCollector versions of the three default plugins that come with LS Data Director 3.2.112 and later. These plugins work the same as the three default ones, except they are to be used with LS Central version 17.5 and later where table names have LSC prefix added.

  6. Click the Add Dest button to add this destination to the list. You can add as many destination databases as you want, and the Monitor data will be saved into each database you define here.
  7. To remove a destination, click the destination in the list, and the click the Del Dest button.

You can also test the connection by clicking the Test Conn button. To be able to test the connection, the data for the destination needs to be in the edit fields. If you are testing a destination that is already in the list, double-click it and the data will be loaded into the edit fields.

Note: When done, remember to click the Save Config button to save the changes locally, or the Send Config button to send the configuration to Remote or running DD Host.

If the setup is large, a lot of Stores and POSs and few DDs running at HO, it can be useful to set up two sets of Monitor Data Collection tables. This can be done by creating two different companies in the Database where Monitor Data should be inserted, and by sending HO DD Data to one company and Store DD Data to another.
By doing this, it is possible to match the data coming from Stores against data coming from HO as there could be a case where a Status Message was lost, and HO DD is waiting for update, while Store DD say the job is done, so the data coming in from both locations will not override each other as they would if both DDs would send to the same set of Tables.

Note: LS Data Director version 3.2.20 and later sets different status on jobs that are missing status update, instead of Unknown the job will get status NoStatus, which means that DD was not able to update the status for that destination, and to be able to get the final status you need to look at the Job Monitor at the destination DD. Having two sets of Tables for the Monitor Collection Data would help checking cases like this, where you find NoStatus in HO job data, and can look it up in Store job data to see the latest status.